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Abstract | . 

Introduction. This paper investigates techniques used by children in 
year 4 (age eight to nine) of a UK primary school to reformulate their 
queries, and how they use information retrieval systems to support query 
reformulation. 

Method. An in-depth study analysing the interactions of twelve children 
carrying out search tasks in a primary school lesson; including 
observations, search recordings, post-task interviews and teacher 
interview. 

Analysis. The search screen recordings were analysed for task 
performance. The queries were categorised using both Broder's 
taxonomy (2002) and query reformulation type schemes. The interviews 
and observations were analysed inductively using thematic analysis. 
Results. Children reformulated queries by switching information 
retrieval system, extracting information from search results, and by 
using 'did you mean' and auto-complete functionality. The children also 
reformulated queries by using previous queries, correcting errors, and 
making their queries more specific. The type of query reformulation 
support the children used varied according to the type of query entered, 
with children submitting both question queries and broad-based topic 
queries. 

Conclusions. The children mostly reformulated queries based on their 
interactions with information retrieval systems, rather than their prior 
knowledge. Search systems could further support children by (1) tailoring 
tools to the type of query (question or broad-based) entered, (2) making 
suggestions for the narrowing of queries as the children found this 
particularly difficult (3) accommodating children's linear scanning of 
search results as this led to unnecessary query reformulations. 











Introduction 


Users of information retrieval systems formulate queries to search for 
information, and reformulate queries to improve search results or gain new 
information. The process is iterative, with queries reformulated from the 
user's own prior knowledge or with the help of the information retrieval 
system (Huang and Efthimiadis. 200Q) . From an analysis of transaction logs 
from Web search engines, over one third of all queries submitted are 
reformulations (Jansen, Booth and Spink. 200Q) . 

Although there is an increasing interest in children's search, very little 
research examines the query reformulation behaviour of children (see Related 
Work). This paper is based on an initial study that investigates how children 
interact with retrieval systems to reformulate queries. This involved an in- 
depth study of twelve children's searches during an information and 
communication technology lesson. In particular this paper examines the 
following: (l) how children use more than one information retrieval system 
within a single search task, and (2) how children reformulate queries using 
functionality, such as 'did you mean' and auto-complete, as well as in response 
to reading search results pages. Specifically we address the following research 
questions: 

• RQi: why do children reformulate their search queries? 

• RQ2: how do information retrieval systems support children with query 
reformulation? 

• RQ3: how do children reformulate their queries? 

Related work 

Much of the research into children's information-seeking behavior focuses on 
the differences between child and adult search behaviour ( Marchionini. 
iq8q : Bilal and Kirby. 2002 : Duarte Torres. Hiemstra and Serdyukov. 
2Qio : Gossen, Low and Nurnberger, 2011) and the impact of search task on 
search behaviour (Bilal. 2002) . Query reformulation is usually discussed in 
relation to these areas; however, children's query formulation and 
reformulation has also been considered in relation to the differences between 
success and lack of success ( Marchionini. iq8q : Duarte Torres, et al ., 

2010 : Kammerer and Bohnacker, 2012) . 

The most comprehensive research study into children's query reformulation 
patterns is the examination of children's search behavior from an analysis of 
search engine transaction logs (Duarte Torres.ef al., 2010) . The analysis 
provides a good overview of the types of query reformulation children enter, 
















and the frequency of reformulation. However, as the analysis is only based on 
the data collected in search logs, the reasons for the reformulations cannot be 
adequately explained. Furthermore, search logs by their very nature only 
record the data for one search engine, and the analysis cannot capture the way 
queries are reformulated across different information retrieval systems. In 
observational studies, research has also tended to concentrate on how children 
search for information in one search system because the search system itself is 
also being evaluated (Marchionini, iq 8 q : Bilal. 2002 ). However, in real life 
situations, similar to adults, children use more than one search system to find 
information. 

Research into how children use information retrieval system assistance for 
query reformulation, and under what circumstances, is very limited. In part, 
this is because many of the features offered to support query formulation and 
reformulation are relatively new and either unavailable in earlier research 
OVIarchionini. iq 8 q : B ilal. 2002! or not yet adopted by children ( Drum etal .. 
200a). It is only recently that children have been observed using assistance 
(Jochmann-Mannak, Huibers Lentz and Sanders. 2010 : Kammerer and 
Bohnacker. 2012k In this study we use a more naturalistic approach to 
studying query reformulation by children that is able to overcome the 
limitations of using only transaction logs and of focusing on search sessions 
based on interaction with single information retrieval systems. 

Method 

In this section, the rationale for the study approach, the sampling procedure, 
the search task, the data collection and analysis methods, and the limitations 
of the study are discussed in detail. 

Rationale 

In observational studies, children are usually studied singly, with search tasks 
recorded outside of the classroom/lesson (for example. Marchionini 
iq 8 q : Bilal 2000a . Bilal. 2000b : Bilal. 2002 : Madden. Ford. Miller and Levy. 
2007 : Foss „ 2012k However, although observation is a key technique for 
understanding user behavior, observation can only shed light on users' 
behaviour if it is real-life behaviour that is observed ( Martzoukou. 2005) . For 
this reason, this study takes a naturalistic approach to data collection. 

The information and communication technology teacher of a Sheffield (UK) 
primary school agreed to the recording of children's searches in his lesson. The 
researcher had no input into the planning of the search task or lesson, and as 
much as possible tried to remain anonymous during the class. However, the 




















children (including those not in the research group) were interested in the 
study so it cannot be claimed that this research did not have any impact on the 
children's behaviour. 

The children, in common with most other primary school children, worked in 
pairs when using technology in schools. Although it is more usual to research 
children's search tasks individually, it was considered more important to 
collect the data naturalistically, and include observations of children's 
collaborative behaviour in the data collection. 

Sampling 

Purposive sampling was used to determine the year group of the children to 
study, and the children selected from the year group. Following a discussion 
with the ICT teacher, Year 4 children (aged eight to nine) were chosen on the 
basis that they were the youngest age where they could mostly search for 
information independently and, therefore, the recorded searches would 
represent the searching behaviour of the children, not the supervising adults. 

The school has two Year 4 classes and it was decided that six children would be 
selected from each class as two smaller groups would be easier for the 
researcher to observe. Furthermore, when testing the screen recording 
software it became apparent that the school network connection slowed as 
more computers ran the screen recording software. 

The participants were selected by the class teachers: the teachers explained the 
research to the children, and pupils interested in taking part placed their 
names in a hat. Each teacher drew six children's names. Letters detailing the 
research were sent home with the children, and the parents were asked for 
their consent. Consent was received for all twelve children selected (seven girls 
and five boys). Immediately before the recording of the children's searches the 
researcher also explained the purpose of the research to the children, and the 
children were shown the recording equipment. The children's verbal consent 
was sought and received before all data collection exercises. 

Search task 

The research was conducted in a Sheffield (UK) primary school with search 
task data collected during two ICT lessons that ran consecutively. For each 
lesson, six pupils aged eight to nine worked in pairs to search for the answers 
to three questions. The questions were chosen by the class and were based on 
the topic the children were learning about that term, the human skeleton. At 
the start of the class the teacher explained to the children the importance of 


safe searching and the need for reliable information. The children then spent 
between ten and twenty minutes searching for the answers, with the time 
dictated by the length of the lesson and the need to set up computers within 
the lesson time. 

Data collection 

Data were collected from observations, recording of search tasks, group 
interviews with the children and an interview with the teacher. 

Observations 

Observations of the children's collaborative and physical behaviour were 
recorded by the researcher in a notebook. In practice, there was little time to 
record observations as the research took place; instead, observations were 
recorded immediately after the searches were finished. 

Screen recordings 

The search task was recorded using Camtasia screen recording software. Prior 
to data collection, Camtasia was temporarily installed on the school 
equipment. 

Interviews with children 

To allow for initial analysis of the search recordings, the children were 
interviewed a few days after the search task. The children were shown 
screenshots of their searches and were asked (l) how they thought search 
engines worked (2) the purpose of search tools (3) the reason for the phrasing 
of their queries (4) what they found easy and what they found hard and (5) if 
they had a magic wand what it would do. Because children may find it difficult 
to express their experiences to an adult f Livingstone. Qlafsson and Haddon. 
2013) the children were interviewed in two groups to boost their confidence. 
The first group consisted of four children from the first class, and the second 
group consisted of the six children from the second class plus the remaining 
two children from the first class who had previously been unable to attend. As 
it happened, the children were confident communicators and more than able 
to explain their experiences. Most of the children were very keen to provide 
explanations, and the hardest part of the interview was to give each child time 
to answer before another child jumped in. 




Interview with information and communication 
technology teacher 


A week after interviewing the children, the teacher was also interviewed. The 
time lapse allowed for some preliminary analysis of the children's search 
behaviour and the children's explanations of their search behaviour. The 
interview questions were semi-structured and the teacher was invited to 
comment on the preliminary analysis. The teacher confirmed the analysis but 
suggested that in general children from the school have above average 
language skills so find query reformulation easier age for age than in some 
other schools the teacher has taught in. 

Verification 

Verification of the analysis was sought from the school, and both the teacher 
and headmaster were given a written report of the findings. No formal 
feedback was obtained, but the findings were informally discussed with one of 
the class teachers and the teacher. 

Data analysis 

The data were analysed both qualitatively and quantitatively with the 
emphasis on the qualitative analysis to produce a holistic and in-depth 
explanation of the children's search behaviour (Mason, 2002k 

Screen recordings 

Initially, the screen recordings were analysed according to the time taken to 
complete the tasks, the impact of queries on the search results, and the use of 
assistance. Then, all of the queries were categorised using two classification 
schemes. Firstly, queries were categorised 

into informationaland navigational queries using Broder's (2002) taxonomy. 
Informational queries are those used to find information relating to the search 
task question; navigational queries are those used to find known sites. 
Secondly, all queries were categorised by query reformulation type based on 
prior schemes (Jansen, Booth and Spink. 2000 : Jesper, Clough and Hall, 
2012k New (first query for search question); Assistance ('did you mean' or 
auto-complete used); Specialisation the query contains more terms or uses 
more specific terms); Generalisation(the query contains fewer terms or uses 
broader terms); Revision (a spelling mistake or grammatical error has been 
corrected); Previous (the query has been submitted before). 







Children's interviews 


The children's interviews were coded inductively using thematic analysis 
(Braun and Clarke, 2006). The initial codes were identified from the data, 
rather than the research literature, and three broad themes identified: (1) 
information retrieval system differences, (2) the phrasing of search questions, 
and (3) choosing between answers. 

Information and communication technology teacher 
interview and children observations 

The teacher interview and search observations were analysed inductively for 
initial codes. The data set was small so the initial codes were not developed 
into themes; instead, the initial codes were incorporated into the themes found 
in the children's interviews. 

Limitations 

Recording detailed observations of children's collaborative and physical 
behaviours by hand by a single researcher was impractical. An overview of 
their behaviour was obtained, and confirmed with the teacher, but for a more 
detailed analysis other data collection methods need to be considered. 
However, the low key approach meant that the children were relaxed about the 
research. The addition of more recording equipment or more researchers 
would have improved the data collection but would have resulted in a less 
naturalistic study. 

The chosen method allows for a deep understanding of how children may use 
information retrieval systems in real-life, but while the findings apply to 
twelve children at a Sheffield (UK) primary school, as with other studies based 
on observation the results are not generalisable (Kelly. Dumais and Pederson. 
200Q) as the sample is size is small and only one task was analysed. 
Furthermore, children at a Sheffield (UK) primary school perform above 
average (Qfsted. 2011) and so the results are not necessarily comparable to 
children in the same year group at other schools. 

Results and discussion 

Task performance 

During the search session, the children could choose up to three search 
questions to answer. However, four pairs only searched for one search 





question, and the remaining two pairs for two questions. In research involving 
transaction logs and observations, children were found to enter fewer new 
queries than adults ( Duarte Torres. Hiemstra and Serdyukov, 2010) and take 
longer to complete tasks ( Bilal and Kirby. 2002k In this study, the lengthy 
time taken to complete search tasks can in part be attributed to the time taken 
to enter queries (for example, the query 'how long is the spine in the average 
human body' took over two minutes to enter) and the children's reading skills. 
However, another reason for the lengthy task completion time was that the 
answers to some of the questions were not easy. Unlike with researcher set 
tasks, the teacher was unable to check the availability of answers before the 
lesson, because the questions were decided during the class. 

Four pairs used auto-complete and three pairs used 'did you mean' assistance. 
In the group interviews, all the children demonstrated knowledge of these 
tools and they discussed the different ways in which these tools could be used. 
All the children started their searches in Google with five of the six pairs 
additionally entering searches in Bing, Wikipedia or Dictionary by Farlex. 

None of the children could touch type so to enter queries they needed to look 
for letters on the keyboard. In some studies, the need to look at the keyboard 
resulted in children only looking at the keyboard and therefore being unable to 
interact with auto-complete (Drum et al, 200Q) , but in this study all of the 
children did periodically look at the screen for this functionality while typing 
(Jochmann-Mannak. Huibers. Lentz and Sanders. 2010) . 

All the children worked in pairs and took it in turns at the keyboard, swapping 
over if (and when) a search question was answered. It was observed in each 
pair that the child who typed the query decided the query. This contrasts with 
their behaviour in other areas of the search where the children did confer, 
notably, which information retrieval system to use, how to spell words, the 
meaning of words, and location of results in the search results pages, although 
even then ultimately the child at the keyboard controlled the search. This is 
similar to studies of adults where finding is a solitary process, but needing and 
using can be a collaborative activity ( Toze, Peet and Toms, 2011) . 

Query formulation 

Altogether, nine navigational queries and twenty-six informational queries 
were submitted. In the group interviews, the children distinguished between 
navigational and informational queries, with navigational queries described as 
easier than informational queries: 'if you were trying to find a specific thing 
you just type in like if you were trying to find out about a person you could 







just check, type their name in and then it might find them' (P12). 

Although the task was informational in nature, three pairs of children entered 
navigational queries to find known sites within which they could search for 
information. During the search task, only one pair directly typed a website 
address into the address bar; in all other cases children navigated to known 
sites by entering the name of the site in the search box. Two of the pairs of 
children entered navigational queries at the start of the task to go to 
Wikipedia, and during the task, two pairs of children used navigational queries 
to change information retrieval system before reformulating informational 
queries, with those children iterating between navigational and informational 
queries. 

From the group interviews it was clear that all of the children put considerable 
thought into the phrasing of queries and how the phrasing affected the answer. 
It was not something they always found easy. When asked 'if you had a magic 
wand, what would it do', two of the children wanted help with the wording of 
a query: 'If I were finding it hard to look something up then I'd just tell the 
computer to find the best words to know to look something up and then it 
would find me the best words that would be easier to make my search' (P7). 
The children were also concerned with how much detail to put in a query. One 
child explained 'because if you just put what is the longest bone it could be 
what if it were an elephant or a cheetah' (Pi). The ICT teacher also 
encouraged the children to be specific with their queries as they will be less 
likely to come across inappropriate material with more defined queries. 

Examining search results 

All children had some difficulty finding information in the results pages 
(Large. Beheshti and Rahmen. 2002 : Bilal. 2000a : Large and Beheshti. 2000) . 
They frequently missed relevant information, and found it hard to choose 
between search results. In the interviews, one child complained of the search 
results page 'there's like a really long list' (P3), and for five pairs of children 
relevant or related information was available within the search results that 
they did not see. These children were able to enter effective searches for 
questions; they just could not see the answers. Children's failure to extract 
information has previously been linked to information overload ( Bilal. 

2000a : Kammerer and Bohnacker, 2012) and loss of focus on search task 
(Marchionini, iq8q) . However, from observing the children in this study 
another explanation is apparent. Using mouse movements as an indicator it is 
likely that all of the children linearly, rather than selectively, scanned the 
search results page. The mouse always moved from left to right, from top to 









bottom of screen. Because the screen was read systematically relevant 
information further down the screen was missed, and this resulted in 
unnecessary query reformulation. For example, one pair on seeing the answer 
for dinosaur bones in the first search result reacted with the following: 'doh, 
we need to write human skeleton' (P2). But the required answer was actually 
in the search result immediately underneath. 

In the group interviews, the children discussed finding it hard to choose 
between search results and they were concerned about the reliability of 
answers: 'if you were looking something up you might want to check that 
because say you looked on Wikipedia, they might be wrong' (P7). These 
children were also aware that performance is different for each retrieval 
system, and one child explained: 'if you use different search engines then 
you'll find different things' (P9). 

Research Question 1: why do children reformulate 
their search queries? 

Altogether there were three reasons for the children's query reformulation: 
they could not find the required information, they wanted to validate 
information found, or they wanted to correct queries. 

Only three of the informational queries were reformulated because the 
information was not available. Five of the six pairs of children reformulated 
queries, despite relevant or related information appearing in the search results 
page. The reason, in this study, was connected to the linear scanning of results 
as the children reacted to information in the order in which it appeared. Only 
one pair checked or verified their answer. They did so by using another 
retrieval system: swapping from Google to Bing and entering a near-identical 
query. Three pairs also revised their queries by accepting 'did you mean' 
suggestions and two pairs corrected their queries for spelling mistakes when 
the correct spelling was identified in the search results. 

Research Question 2: how do information retrieval 
systems support children with query reformulation? 

From an examination of the informational queries, it is apparent that two very 
different types of informational queries were entered. Four pairs slightly 
adapted the original task questions and entered queries as specific questions. 
Two pairs, instead of basing the queries on the search task, entered queries 
based on the broader class topic. For example, for the search question ’what is 
the longest bone' one pair entered the query as a specific question, 'what is the 


longest bone in the skeleton', and another pair entered the query as the 
broader class topic ' human body'. 

In a study that compared children's keyword and natural language queries, 
children were marginally more successful when using natural language queries 
because the search results were more applicable to the search task questions 
('Kammerer and Bohnacker. 2012k and in this study answers to the questions 
were often available within the search results for the question queries but the 
results needed to be examined for the broad topic queries. However, there 
were also other differences between the two query types, as the different styles 
of queries resulted in different query reformulation behaviour with 
information retrieval assistance used in different ways. 

Question queries 

When employing question queries, the children used the retrieval systems for 
support in query reformulation by applying the 'did you mean' suggestions 
and extracting information from the search results. Auto-complete was not 
used for this purpose. 

Three pairs of children reformulated queries based on their reading of the 
search results pages: queries were corrected for spelling mistakes, and also 
refined to make use of information found. Three pairs of children accepted 'did 
you mean' suggestions in the search results page. For example, for the query 
'what is the longest bone' the 'did you mean' synonym suggestion of 'what is 
the largest bone' was accepted. For the question queries, auto-complete was 
only used to correct spellings and provide typing shortcuts. From the 
recordings, it is apparent that the wording of the queries had been pre¬ 
planned and auto-complete was only selected when it matched the exact 
wording of the planned query, with other appropriate suggestions ignored. For 
example, for the planned query 'how many bones in a human babys foot', 
auto-complete made alternative suggestions, but because none of the 
suggestions exactly matched the planned query, auto-complete was not used. 

Broad topic queries 

The two pairs of children entering broad topic queries did not reformulate 
queries based on their reading of the search results pages, and the retrieval 
systems did not offer 'did you mean' suggestions. The two pairs used auto- 
complete to assist both the initial query formulation and subsequent 
reformulations. In the group interviews, it was apparent that some of the 



children seemed to associate retrieval systems with either having or not having 
information (for example, 'some of the search engines might not have the 
question, might not have the answer to your question' (P4)) rather than 
linking results to differences in query formulation. This may account for why 
auto-complete was used to suggest queries. 

Research Question 3: how do children reformulate 
their queries? 

Of the informational queries, 8 (31%) were classified as New; 7 (27%) 
Assistance; 4 (15%) Specialisation; 3 (12%) Previous; 2 (8%) Revision; and 1 
(4%) New/Assistance. No Generalisation type was used and one query 
reformulation was submitted by mistake. There were no reformulations of the 
navigational queries. In the same way that children used different types of 
support depending on the type of query entered, how the children 
reformulated the query also varied depending on the query type. 

Navigational queries 

In this study, none of the children reformulated navigational queries, although 
one might expect to see this if the navigational queries were difficult to spell or 
the name of the site the children intended to visit was not fully known. 
However, two pairs of children did iterate between navigational and 
informational queries, and after submitting a navigational query the next 
informational query was a Previous query. In fact, all three of the Previous 
queries were submitted in a new information retrieval system. That children 
resubmit previously used queries is well known (for example. Gossen , Low and 
Nurnberger. 2011k but this is usually linked to children's cognition and recall 
difficulties (Bilal and Kirby. 2002I rather than an intentional technique. This 
was clearly a technique other children used too, and when questioned over 
whether it was necessary to use another retrieval system to check information, 
a child wisely pointed out 'well you could check on the same but on a different 
search engine you'll get a wider range' (P9). 

Question queries 

The children made Assistance, Specialisation and Revision type 
reformulations when their queries were questions. Five of the Assistance 
reformulations were made following 'did you mean' suggestions. The 
suggestions included spelling error corrections, synonym change and 
broadening queries. 





In most instances the children were concerned to make their question queries 
specific, and notably queries were not generalized except for when following 
'did you mean' suggestions. As well as reformulating queries to make them 
more specific, the children also increased the specificity of the search task 
questions in the initial New queries. However, the implementation of 
Specialisation was often misguided. For example, one pair tried to use their 
topic knowledge to increase specificity and added ' mans average foot' to the 
task question 'how many bones in your foot'. While adding mans does 
increase the specificity because a baby's foot has more bones (and the pair 
knew this), adding 'average' does not increase the specificity because adult 
feet typically have the same number of bones. The specificity of queries was 
also increased by including information found in the search results. This was 
also problematic. For example, one pair changed 'spine' to 'spinal cord' in 
'how long is the spine in an average adult' but the length of the spine and 
spinal cord are not equivalent. Analysis from the study of a transaction log 
found that it was more common for children to add rather than remove words 
from a query, but reducing the number of words was more effective. However, 
all query reformulation types (except for Previous Query) led to children 
selecting higher ranked results (Torres. Hiemstra and Serdyukov. 2010) . 

For Revision type reformulations, two pairs of children revised their queries, 
based on their readings of the search results, to correct grammar and spelling 
mistakes. For example, despite seeing relevant results, one pair realised there 
was an error in their query and changed 'how many bones in a human babys 
foot' to 'how many bones in a human babies foot', ironically introducing 
another error. The revisions were not necessary and did not improve the 
search results. However, the children were in a school lesson where they are 
assessed on spelling and grammar so it is understandable why the corrections 
were made. During the interview with the ICT teacher the importance of 
spelling was stressed in terms of speed (7 say when you are doing a search 
make sure your spellings are correct so it will make your search quicker') and 
Internet safety in case a misspelling led to inappropriate results. 

Broad topic queries 

The children only used Assistance type reformulations when their queries were 
based on the broad topic, and they only used auto-complete to do this. In all of 
these cases the children used auto-complete to specialise their queries. For 
example, one pair started their search by typing 'human skeleton' into the 
search box and then selected 'human skeleton for kids' from auto-complete. 
After examining the search results page with no success, they again typed 



'human skeleton' in the search box and then this time selected 'human 
skeleton facts' from auto-complete. 

Conclusions 

This study investigates the query reformulation patterns of children based on 
an in-depth study of twelve children carrying out a search task in the 
classroom. The findings suggest that there is a clear role for information 
retrieval systems to provide support for children's query reformulation: the 
children used prior knowledge to formulate queries but mostly reformulated 
queries based on their interactions with the information retrieval systems. To 
do this the children used a number of techniques, including retrieval system 
switching, examination of search results, and use of "did you mean" and auto- 
complete functionality. However, more tailored support could be offered. 
Firstly, the children had the most difficulty with, and need the most support 
for, narrowing their queries. Secondly, there were two distinct types of 
information queries (broad topic and questioning queries) which resulted in 
different usages of search tools, and these tools could be developed in 
accordance with how the children are using them. Thirdly, more could be done 
to accommodate children's linear (rather than selective) scanning of search 
results as this resulted in unnecessary reformulations. 

A further collection of data is planned to see whether these findings are also 
seen across a larger dataset, how query reformulations vary by search task 
type, and whether query reformulation patterns change when a search task 
spans multiple search sessions. 
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